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ABSTRACT 

In June 1993, the Quebec Commission d' Evaluation de 
1 'Enseignement' Col legial (CEEC) was created to conduct evaluations of 
the assessment of student achievement and quality of college programs 
of studies in the province. This report presents, in two parts, CEEC 
guidelines for evaluating programs of studies. Part 1 describes the 
CEEC and educational renewal in Quebec, highlighting the legal duties 
and responsibilities of the Quebec Minister of Education in 
determining program standards, the colleges in implementing 
authorized programs, and the CEEC in evaluating institutional 
policies on program evaluation. Part 2 discusses the CEEC's 
objectives and methodology in evaluating programs of study and their 
implementation, describing the following steps in CEEC evaluations: 
(1) the evaluation guide and criteria will be, adapted to each program 
of studies to be evaluated; (2) using the CEEC guide, each 
institution will conduct its own evaluation of the program and submit 
a report to the CEEC; (3) the self-evaluation report will be reviewed 
and site visits conducted; (4) a preliminary evaluation report will 
be prepared and distributed to the institution for response and 
corrections; and (5) a final evaluation report will be prepared. 
Finally, the criteria for program evaluations are presented, 
including program relevance, program coherence, the value of teaching 
methods and student supervision, the appropriateness of resources 
allocated to educational needs, the effectiveness of the program, and 
the quality of program management. (BCY) 
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Introduction 



The Commission devaluation de Tenseignement coltegial was created by Bill 83, assented 
to in June 1993. 1 An autonomous, independent body, it is mandated to evaluate, i.e. "rule 
formally on, how colleges fulfil their academic responsibilities" 2 by evaluating the content 
and implementation of institutional policies the evaluation of both student achievement and 
programs of studies. 

To conduct its evaluation of college programs, the Commission has defined its evaluation 
methods and criteria 3 that will serve as the basis of its judgments. It is tense to inform 
all those interested in the question of evaluation in college education, in particular the 
colleges themselves, of the methods and criteria it has chosen. 

This document, intented mainly for college staff, has a prelimanary status. Enriched by the 
comments of those to whom it his intended, il will be finalize over the coming months. 

Herein the Commission presents the general guidelines underlying its approach to program 
evaluation. The document is divided into two parts: Part One discusses the Commission's 
mandate and Part Two, its evaluation methods. 



1. Act respecting ihe Commission devaluation de I enseignement colUgial and amending certain legislative 
provisions. Bill 83 (1993, chapter 26). 

2. MESS, Colleges for the 21st Century (Quebec, April 1993), p. 40. 

3. The Commission will publish a separate statement of general guidelines for the evaluation of institutional 
policy on program evaluation (IPPE). 



Part One 

The Commission's Mandate 



1. Contributing to the Renewal of College Education 

The creation of the Commission devaluation de I'enseignement colldgial is an important 
measure contributing to the renewal of college education. It reflects the converging will of 
the Minister of Education, the colleges, and the vast majority of organisations and 
associations in college and socioeconomic circles to recognize the importance of college- 
level institutions in higher education and provide them with the tools they need to develop 
education at this leveL As a result of the renewal, colleges will assume more 
responsibilities in this regard, and internal and external evaluation mechanisms will become 
more reliable and effective. Policy and program evaluation should improve the quality of 
procedures and, ultimately, owing to the critical reflection fostered by the evaluation 
process, raise the calibre of college education and make it more relevant. As the 
Commission fulfils its mandate to evaluate institutional policies and the implementation of 
programs of studies, the value of a college education and the diplomas colleges offer 
should receive greater recognition. 

The Commission will contribute to the renewal of college education by conducting 
evaluations aimed at guaranteeing and attesting to the quality of student achievement and 
programs of studies at this level 

2. Evaluating Programs of Studies 

In evaluating college programs, the Commission will seek to certify their relevance and 
affirm the quality of their implementation by college-level institutions. In so doing, it hopes 
to help improve these programs in order to ensure "all Quebecers of access to a high- 
calibre, top-quality college education that enables them to attain the highest possible 
skills standards." 4 



4. MESS, Colleges for the 21 si Century (Quebec, April 1993), p. 19. 



The Commission intends to make its program evaluations available to those involved in the 
college sector and to the public as a whole to enable people to form as accurate an idea as 
possible of the evolution and quality of college instruction and the results obtained. 

3. Regulatory Framework 

The regulatory framework for college program evaluation is based primarily on the Act 
which created the Commission, i.e. the Act respecting the Commission devaluation de 
V enseignement coltegial and amending certain legislative provisions (Bill 83). However, 
in conducting its evaluations, the Commission must also comply with various other statutes, 
rules and regulations pertaining to the elaboration, implementation and review of college 
programs. This section will highlight the major provisions governing the activities of the 
Ministry, the colleges and the Commission itself as stipulated in the General and 
Vocational Colleges Act, the Act respecting private education, Bill 83, the College 
Education Regulations, and the decision handed down by the Minister of Higher Education 
and Science on September 15, 1993. 

3.1 The Minister's Duties and Responsabilities 

The Minister is responsible for defining the overall organizational framework for college 
education, most notably as far as programs of studies are concerned, it determines the 
objectives and standards for programs leading to a Diploma of College Studies (DEC), and 
provides the financial resources that are required to implement these programs. 

3.2 The Colleges' Dyties and Responsabilities 

Colleges have as principal responsability the implementation of the programs for which 
they have received the Minister's authorization. 5 These programs lead to a DEC in the 
pre-university and the technical di mains. In subject areas related to technical studies, they 
may, under certain conditions, design and implement programs leading to an Attestation 
of College Studies (AEC). 

Based on recent amendments ;c the General and Vocational Colleges Act, the academic 
council is responsible for advising a college's board on any matter concerning programs 
of studies, in particular on institutional policy on program evaluation, proposals for 



5. General and Vocational Colleges Act, s. 6, first paragraph, subparagraph a. 
4 
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programs of studies and the selection of learning activities that are within the jurisdiction 
of the college. 6 

Divisions HI to V of the College Education Regulations refer to programs of studies, 
defining the colleges* responsibilities in this regard, including those related to the renewal 
of college education. Colleges must determine which learning activities will make up the 
general education component specific to each program and the general education 
component that is complementary to the other components of a program. They must also 
determine the learning activities of a specific program component of technical studies and 
at least 50 percent of the specific program component of pre-university studies. In addition, 
they must adopt and make public a description of the objectives, standards and learning 
activities for each program offered. After consulting with the academic council, the colleges 
must adopt and implement an institutional policy on program evaluation (IPPE). 

Unlike the procedure applicable to the formulation of an institutional policy on the 
evaluation of student achievement (IPESA), whose content is determined by specific rules 
and regulations, the Ministry has entrusted each college with the task of defining the 
content of its institutional policy on program evaluation. 

3.3 The Commission's Duties and Responsabilities 

Under chapter II of Bill 83, the Commission is responsible for evaluating the content and 
implementation of the IPPE adopted by each college, the implementation of programs 
whose standards and objectives have been set by the Ministry, and the objectives, standards 
and implementation of programs established by colleges, based on the needs these programs 
are intended to meet. The Commission may also evaluate the implementation, by all or 
some of the colleges, of any program of college studies it designates. It may exercise 
auditing powers in carrying out its mandate. 

The Commission is required to make its evaluation reports public in the manner it 
considers appropriate and may, where necessary, make recommendations to the educational 
institution concerned and to the Minister. 



6. Act to amend the General and Vocational Colleges Act and other legislative provisions, s. 17.01 and 
17.02. 



4. A Progressive Approach Involving Ongoing College Participation 



Given the importance of its mandate and its desire for constructive action, the Commission 
favors a progressive approach which takes into account the wide range of experience 
acquired of colleges in evaluating programs of studies. Both the Commission and the 
colleges face major challenges. The latter must formulate, adopt and implement an IPPE 
and methods for conducting their own evaluation of the programs they offer. And, the 
Commission must evaluate the content and application of these institutional policies and 
the implementation of programs of studies. 

The Commission believes that it should be relevant to evaluate a few programs of studies 
very soon. This will involve the participation of both the colleges offering the programs 
and external specialists known for their expertise in the subject areas concerned, and would 
allow the Commission to test its approach to program evaluation in order to determine 
whether it is realistic and readily applicable to all types of programs, to make valid 
judgments on the content and implementation of programs, and to identify major problems 
and possible causes. 

By actively participating in testing the Commission's approach, the colleges would be able 
broaden their experience in program evaluation while deriving inspiration for the IPPEs 
they must formulate and adopt. 
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Part Two 

The Commission's Approach to Program Evaluation 



1. Objectives 

In evaluating programs, the Commission will seek to ensure that colleges offer high-quality, 
relevant education. It will also seek to promote recognition for the q uality and relevance 
of college education among all stakeholders, including the students themselves. 

The notion of quality is reflected in the relationship between course contents, education 
process, and results and predetermined objectives which corresponds to needs. The needs 
are expressed by society, namely by employers, educational institutions, professional 
associations, students, academic peers, administrators, and financial backers. These 
expectations affect to numerous aspects of the system and of learning activities within it: 
the skills and knowledge acquired by graduates, the value of model of teaching, the quality 
of student supervision and guidance, the coherence of the curriculum, the competence and 
enthusiasm of the teaching staff, the availability of material resources and support services, 
the efficiency of management, etc. Quality cannot be defined in unequivocal, absolute 
terms, since it takes on different meanings depending on the field of study or the type of 
program under consideration. 

The concept of relevance, whifch coincides with that of quality, refers to the extent to which 
programs are likely to meet socioeconomic, educational, technical and cultural needs. 

Given the fact of the expanded autonomy which has been given to colleges in 
implementing programs, the Government has clearly expressed its will to counterbalance 
these new responsibilities with what are intended to be more effective evaluation 
mechanisms, e.g. the program evaluations conducted by the Commission, which will be 
made public. The Commission thus intends to play an important role in attesting to the 
quality of college education. 

2. Evaluation Method 

The Commission will evaluate a program or a group of programs leading to a Diploma of 
College Studies (DEC) by examining how they are implemented in each of the colleges 
offering them. Generally, it will evaluate programs leading to an Attestation of College 
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Studies (AEC) on a case-by-case basis, examining not only how they are implemented by 
the college concerned but also their objectives and standards taking into account the needs 
these programs are designed to meet. The Commission may adapt its approach to take 
special circumstances into account. 

In conducting its evaluations, the Commission plans to use the means, resources and 
procedures of evaluation with features inspired by and adapted from those generally found 
in the area of post-secondary program evaluation. Finally, a report will be prepared at the 
end of each program evaluation and made public in the manner the Commission considers 
appropriate. 

2.1 Features of the Evaluation Process 

The characteristic features of the Commission' 3 evaluation process may be summarized by 
the following three statements: 

1) The program evaluations conducted by the Commission will require the participation 
of both persons involved in implementing programs at the local level, Le. in the 
college itself, and external experts. 

Local participation consists of a self-evaluation through questions provided by the 
Commission. This self-evaluation reinforce in particular the formative character of program 
evaluation and paves the way for the examination of the program by the Commission. 

In calling on the contribution and judgement of persons with known competencies in the 
domain covered by the evaluation and who do not participate in local implemention of the 
program, the Commission carries out its external evaluation while ensuring at the same 
time proper distance and a widening of perspectives. This will enhance the quality, 
credibility and objectivity of the process as a whole. 

2) The Commission will exercise its power to make recommendations by suggesting 
means to improve the quality of education and by evaluating the relevance of 
programs of studies. 

Following the evaluation of a program of studies, the Commission writes a report on its 
findings, including recommendations to the college concerned and to the Minister of 
Education. 
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The Commission will strive to highlight the strengths and weaknesses of a particular 
program and suggest ways which will reinforce the strengths or correct the weaknesses 
identified. If necessary, the Commission counts on the cooperation of colleges to adress 
recommendations to the Minister aimed at making DEC programs more relevant. In 
borderline cases, the Commission may recommend to the Minister thai a DEC program 
acknowledged to be nonrelevant, owing to outmoded objectives and standards, be 
withdrawn or that a college no longer be authorized to implement a program unless it can 
offer minimal guarantees for the quality of collegial teaching. 

3) The Commission will make the results of its program evaluations public. 

The Commission has opted for an approach based on transparency. It plans to disseminate 
the results of its evaluations in order to testify publicly to the manner in which the colleges 
concerned fulfil their mandate to offer high-quality instruction. 

2.2 Means and Resources 

The Commission will distribute an evaluation guide identifying the basic data required, 
evaluation criteria, the norms on which it will base it evaluations as well as a series of 
questions which are designed to help the institution carry out its own self-evaluation. 

Advisory committees will assist the Commission in program evaluations. Each committee 
will be chaired by a member of the Commission and made up of persons with expertise 
from the field of teaching, the businc ss world and socioprofessional organizations. The 
number of members will vary up to a maximum of seven, depending on the number of 
colleges offering a given program or the number of programs being evaluated at one time. 
Committee members will be selected from a list drawn up by the Commission following 
consultation of the socioeconomic and educational organizations concerned. 

The Commission may call upon the assistance of experts at any point in its work. They will 
be invited mainly to sit on one of the advisory committees or to give their opinion on a 
particular aspect of the program evaluation under way. 
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2.3 The Evaluation Procedure 

The evaluation method envisaged by the Commission includes the following steps: 

1) With the assistance of the advisory committee, the Commission will adapt the evaluation 
guide and criteria to each program of studies it intends to examine. It will send the 
guide, criteria and other pertinent information to those authorized in each of the 
institutions concerned. 

2) Each institution will conduct its own program evaluation using the guide provided by 
the Commission. The institution will submit a self-evaluation report to the Commission 
along with all other documents it considers relevant. The report must identify the 
strengths, weaknesses and development potential of each program under review. If it 
expects to undertake any action, the institution must inform the Commission of this. 

3) The Commission will study the self-evaluation report submitted by each institution. It 
will then pursue its evaluation by visiting the college in order to discuss certain aspects 
with the institution and obtain additional information. The evaluation visits will normally 
be conducted by tliree members of the advisory committee, one of whom must also be 
a member of the Commission. 

4) The Commission will prepare a preliminary evaluation report for each of the programs 
and institutions concerned. It will send a copy to the institution to allow the institution 
to point out any errors or omissions of facts, react to its content, provide explanations, 
make comments and, if need be, indicate any measures it intends to take in order to 
improve the situation. 

5) The Commission will prepare a final evaluation report taking into account the comments 
submitted by the college. 

2.4 Reports and Recommendations 

The Commission will prepare two types of reports: 

1) a program evaluation report for each institution concerned. Its recommendations will be 
an integral part of this report. 
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2) a program evaluation report summarizing its conclusions for all of the institutions 
offering the program under consideration. Where needed, it will also adopt 
recommendations that will be included in this report. 

2.5 Publication 

The Commission will disseminate its reports as follows: 

1) It will send a copy of the evaluation report to the institution concerned and to the 
Minister. 

2) It will send a copy of the report summarizing its conclusions to each of the institutions 
concerned and to the Minister. 

3) It will make these reports public ir, the manner it considers appropriate. 
3. Evaluation Criteria 

The Commission has its role to evaluate n the implantation of the programs of studies 
established by the Minister of Higher Education and Science, taking into account the 
objectives and standards assigned to them 11 as well to evaluate "the objectives, standards 
and implementation of the programs of studies established by the institution taking into 
account the needs these programs are designed to meet. 7 ' 1 

In evaluating programs, the Commission will focus on the objectives, resources, teaching 
methods and results of education. The Commission retains six criteria on which it bases 
its judgement about the value of a program. Each criterion may be broken down into a set 
of norms all of which indicate what the Commission considers to be minimal guarantors 
of a program's relevance and the quality of its implementation. 



7. Ibid., s. 13. 



3.1 The Relevance of the Program: conformity between objectives, standards and 
program content and the socioeconomic and socioeducational needs 8 

1) The objectives and standards of a program of studies, as established by the Minister or 
the institution, are in agreement with society's expectations and needs. 

2) The objectives, standards and program content take into account, through an ongoing 
analysis, the needs of the labour market and the graduates which are turned out, 

3) The objectives, standards and program content take into account, through an ongoing 
analysis, the expectations of universities, the college graduate access to university, their 
performance, and their satisfaction with their college studies. 

4) The objectives and program content are consistent with the college's educational project, 
the values and general objectives of the institution, including its regional development 
mission. 

3.2 The Coherence of the Program: organization of program structure, content and 
learning activities with respect to the objectives and standards which have been 
assigned to it 

1) The program objectives, as determined by the Minister or the college, clearly describe 
the competencies that students must acquire, as well as the standards establishing the 
level to which these competencies must be mastered by college student, 

2) The program includes a combination of learning activities involving general and 
specialized education whose specific objectives are clearly defined and consistent with 
the objectives of the program. 

3) The content of each learning activity allows program objectives to be achieved. 

4) Learning activities are ordered in a logical way going from the simplest to the most 
complex and sequenced in such a way that facilitates the introduction, in-depth study 
and synthesis of the various objectives. 



8. Note: The first three criteria do not all apply to the evaluation of programs leading io a DEC. However, 
without intending to usurp the role of ministerial authorities responsible for evaluating the relevance of 
these programs, the Commission plans to obtain feedback from the colleges in order to submit, if need 
be, recommendations to the Minister likely to make these programs even more relevant. 

12 
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5) Specific requirements for each learning activity (courses, laboratories, individual 
projects) are realistic and clearly defined; such requirements are accurately reflected in 
course outlines and the weighting and calculation of credits. 

3.3 The Value of Teaching Methods and Student Supervision: means used to reach 
the objectives of the program and those of each learning activity 

1) Teaching methods are adapted to the objectives of the program and those of each 
learning activity, 

2) Teaching methods also take students' characteristics into account in order to help them 
achieve program and learning activity objectives in accordance with established 
standards. 

3) Measures designed to identify learning difficulties as well as guidance, support and 
follow-up activities enable students to overcome these difficulties, 

4) Teachers set up their schedule in such a way as to be able to respond to students' 
questions and needs. 

5) Teaching methods are reviewed on a regular basis to improve them and keep them up- 
to-date. 

3.4 The Appropriateness of Human, Material and Financial Resources According to 
Educational Needs: the quantity and quality of resources allocate to programs by 
the institution 

1) The number and quality of teachers are sufficient and the areas of their expertise are 
diversified enough to meet the objectives and learning activities of the program. 

2) The motivation and competence of teachers and other categories of personnel are 
maintained, among other things, through well-defined evaluation procedures and 
professional development activities. 

3) There is a sufficient number of support staff with the qualifications needed to satisfy the 
needs of the program. 
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4) The quantity and quality of classrooms, equipment and other physical resources are 
sufficient; their access, maintenance and upgrading norms are in conformity with 
educational needs. 

5) Financial resources are sufficient to ensure the proper functioning of the program. 

3.5 The Effectiveness of the Program: results of student learning in relation to 
targeted objectives and standards taking into account the resources allocated to 
programs 

1) Recruitment, selection and integration measures which allow for the intake of groups of 
students who are capable of successfully completing the program. 

2) Specific requirements for each learning activity are effectively respected. 

3) Graduates meet established standards for the acquisition of the competencies established 
for the program. 

4) A sufficient number of students complete the program within the normal time frame, 
taking in account their status (i.e. part-time or full-time student, average number of 
credits per student) and their characteristics. 

5) The success rate in courses is satisfactory and compares favourably with that observed 
in other programs and institutions. 

6) The evaluation of student learning conforms to the college's IPESA, whose content and 
implementation are judged satisfactory by the Commission. 

3.6 The Quality of Program Management: structures and methods of management, 
organizational context, implementation and evaluation of the program 

1) The structures and methods of management as well as the existing means oi 
communication promote the coordination and proper functioning of the program and the 
"program approach". 

2) The division of responsibilities for program planning, organization, management and 
evaluation are clearly defined and fully operational. 
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3) Clear procedures which aid in the regular evaluation, using qualitative and quantitative 
data, of the strengths and weaknesses of the program and each of its learning activities. 

4) Program descriptions are only distributed and explained to students and teachers alike. 
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